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To protect children’s rights and promote their development, quality in early childhood settings is 
essential. Across the global early childhood community, there is consensus that quality is both 
immensely important and unfortunately, not adequately ensured. Reflecting the broad support 
for attention to quality, ISSA, UNICEF, UNESCO and the Bernard van Leer Foundation, in 
partnership with the World Bank and the Brookings Institution, joined forces to call attention to 
the importance of measuring and improving quality in early childhood environments. An 
international consultation was convened in Leiden, the Netherlands in September 2014, bringing 
together early childhood experts and stakeholders representing different regions of the world, 
who have experience in the measurement and improvement of quality of early years services. 
The meeting aimed to identify points of consensus in measuring quality and future directions for 
increasing the strength, innovation and cultural applicability of quality measurement. 


There has recently been increased attention to measuring 
quality on many levels, from independent researchers 
developing methods, to governments and international 
organizations interested in measuring quality at scale. 
Although several new and innovative approaches to 
measuring and addressing quality in early childhood 
settings have been developed in recent years, they are not 
widely or consistently used. The meeting provided a venue for the participants to take stock of 
new ideas in measuring early childhood quality and to explore setting up a global network around 
the issue of quality in early childhood settings and systems. 


Two papers were prepared in advance of the meeting: a summary of existing frameworks on 
quality measurement that were generated by large groups of stakeholders; and a review of the 
existing status of monitoring and evaluation frameworks and their implementation especially in 
low-resource countries. From these reviews, several questions emerged. These questions 
formed the backbone of the discussions during the meeting: 


e What are the points of convergence across frameworks? 

e How can we measure quality in ways that are culturally and situationally relevant? 

e How can we meaningfully measure quality, especially process quality, at scale? 

e How can measures of quality be used within national systems to foster improvement? 
e How can an international network further support improvement efforts? 


The meeting format combined presentations with small group work to discuss what is critical to 
measure; how it could be measured; and what principles or themes should underlie quality 
measurement, especially in regards to how the measures can be used for improvement. 
Presenters were asked to share summaries of their work relevant to the measurement of quality, 
including new measures of early childhood settings and systems developed for use in low-income 
settings; measures of new dimensions of quality, such as the role of nature and architecture in 
quality; and approaches to understanding systems-level impacts on quality, including policy and 
regulatory approaches. 


Points of convergence in measuring quality 


The group was in consensus that quality in early 
childhood settings could be reliably measured and 
on what should be monitored. There was also 
consensus on the need to link setting and system 
quality and that the data collected should be used 
for improvement purposes and not to penalize. 


e What should be measured? Groups were 


asked to reflect on the presentations from 

the meeting and identify areas of quality measurement that could be considered a “core” 
of quality that would cut across all settings including center, school, home and community 
—based provisions. Eight broad areas that should be included in measures of early 
childhood quality were generated: 
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Environment and the physical setting (including infrastructure, safety, cleanliness, and 
enabling) 

Family and community engagement 

Personnel (including initial preparation, on-going training and support, caregivers 
well-being, professional code of ethics) 

Interactions (including between adults and children, between children and between 
adults including staff, supervisors, community, families) 

Structural support (including monitoring and evaluation, financing and logistics) 
Inclusiveness 

Program structure and curriculum 

Health and hygiene (including mental and physical health and nutrition) 


For each of these areas, some groups also generated specific items that should be 
measured, such as the availability of toys and other materials; staff training; and access 
to outside play. Each of the eight areas were conceived as being translated into a set of 
items or indicators that could form the basis of a measurement tool to include universal 
items that are non-negotiable because they are central to protecting children’s rights and 
promoting their development and other items which are equally as important but need 
to be aligned to reflect local values and priorities. A strengths-based approach was also 
emphasized. 


How should setting and system level quality be linked? Throughout the meeting, the 
question of how setting and system-level quality work together was discussed. Setting- 
level quality was defined as referring to what takes place within specific early childhood 
programs. System-level quality, or the impact of policies and regulatory practices on 
quality across many programs, emerged as a key area of focus for future measurement, 
and for which there is a limited number of existing tools. In particular, the group 
articulated a vision for having a set of indicators that applied to both setting and system- 
level quality, to be collected routinely within countries. 


Systems should monitor how ‘generously’ they promote compliance with the Convention 
on the Rights of the Child as the foundation for measuring system and setting quality. 
Systems also need to be monitored in terms of how well they provide for other macro 
level elements such as family and community support which form a protective shield for 
early childhood development and family life quality. 


What methods exist for measuring quality? Across the measures discussed, observation 
was seen as central to quality measurement at the settings level, with innovations in how 
that observation unfolds to include using technology to generate reliable information on 
the overall level of quality. 


Despite the agreement that system-level quality is important to measure, there has been 
less work to date in this area, and thus we are farther from consensus on how it should 
be measured. Most measures to date have relied on outside consultants or government 
reporting on the status of policies and regulations and their implementation. 


How should the data be used? Agreement on key themes on how the data should be 
used emerged as well, with the caveat that more work is needed to clarify how best to 
ensure data lead to improvement: 

o Clarity on the purpose of collecting the data was seen as the first step to good 
measurement. Agreement among all key stakeholders on the role of the data within 
quality improvement efforts must be achieved before data can lead to improvement. 

o A focus on improvement. A core principle emerged on the importance of ensuring 
data are used as formative assessment for improvement. Participants did not want 
early childhood programs and services to be penalized because of inadequate funding 
and lack of professional preparation of those who work in them, and instead, wanted 
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to ensure that the measurement facilitated improvement and greater government 
support (including public funding and technical assistance) and priority on areas of 
need. 

o Data on quality were seen as perhaps best suited for people within the country system 
who are responsible for several early childhood settings and can report on aggregate 
data to policy makers, rather than emphasizing data that are designed separately for 
programs or policymakers. There was agreement that quality measurement would 
be designed to be relevant to both groups. Data should also be openly shared and 
centrally reported including budgets, financing, staff wages/gender, etc. 

© Open-source data with reporting mechanisms for multiple types of stakeholders. 


New approaches in quality measurement 


Three main themes emerged during the meeting on potentially new approaches for quality 
measurement: 


1. There is growing appreciation of the importance of the natural and built environments in 
early childhood settings. Presentations on how to build and measure natural 
environments for young children outlined creative approaches that may be especially 
powerful complements to early childhood settings for children in low-resource settings. 


2. Data should be designed for feasible, effective use at scale. Methods for data collection 
that can help facilitate accurate, feasible measurement at scale include digital imaging of 
early childhood settings (videos, pictures); adaptation of a common framework of items 
to respond to local communities while building on existing measures; and relying on open- 
source tools and data. 


3. Community involvement in defining and measuring quality is critical. There are existing 
models for engaging communities in the dialogue on how quality should be defined and 
measured. The general areas of consensus around quality measurement outlined above 
must be adapted to be fully responsive to community values, which may lead to a 
different set of items and specific indicators in different places. 


A global network on quality 


To capitalize on the wealth of expertise and commitment to quality and to encourage scaling of 
quality, participants in the meeting expressed an interest in establishing a network and 
recommended the following: 


e Enlarge the participation in the network. The inclusion of more stakeholders especially 
those working in low resourced countries and regions would enhance the opportunities 
for shared learning. 


e Develop asystems-to-settings measurement approach. The strong agreement that both 
systems and settings should be included in measures of quality will require development 
of new tools and/or items that can be used to link systems and settings within countries. 
For example, it may be possible to create tools for measuring quality that track both the 
delivery of high-quality service at the setting level, and also track progress towards 
ensuring supportive policies and their implementation at a national or regional level. New 
stakeholders (such as those from the health and social welfare sectors) may be needed 
to ensure that tools are designed for maximum effectiveness in leading to improvements. 


e Focus on monitoring and evaluation systems, in addition to measurement tools. The 
design of a monitoring and evaluation system is crucial to using data for improvement. 
The background paper and the group conversation both emphasized that while quality 
standards may be in place, little is being done to implement them. Data are inconsistent 
and not publicly available, which further slows the progress towards improvement. More 
work is needed to outline how countries can develop effective monitoring and evaluation 
systems that will promote use of data. 


e Quality measurement in emergency and humanitarian crises situations deserves further 
attention. Children in these environments are extremely vulnerable and quality early 
childhood settings can play a crucial role in providing support. But methods must be 
developed that are feasible and designed for use in challenging situations. 


e Develop a common language for quality. How we use and refer to terms, standards, 
principles, elements, components etc. of quality need to be agreed upon. 


For more information on this international consultation and on the plans for supporting a global 
network to improve quality in early childhood settings, contact admin @issa.nl. 


